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SUMMARY 


Problem 


There  has  been  almost  no  increase  in  the  predictive  validity  of  traditional  psycho¬ 
metric  tests  since  they  were  first  put  into  use  more  than  60  years  ago.  Recent 
developments  in  the  cognitive  and  computer  sciences,  such  as  computerized  tests  of 
cognitive  (mental)  speed,  appear  to  have  considerable  potential  for  adding  incremental 
validity  to  existing  aptitude  batteries.  Little  is  known,  however,  about  the  psychometric 
characteristics  of  many  of  these  newer  measurement  techniques. 

Purpose 


The  purpose  of  the  present  study  was  to  determine  the  reliability  (in  particular,  test- 
retest)  coefficients  for  a  battery  of  microcomputerized  cognitive  speed  tests.  In  addition, 
the  validity  of  such  tests  was  explored. 

Approach 

A  set  of  four  reaction  time  tests,  three  inspection  time  tests,  and  an  experimental 
test  of  speed  of  processing  in  active  memory  were  administered  to  104  male  and  female 
college  students  between  18  and  35-years  of  age.  Seventy-four  subjects  returned  for 
retesting  with  10  days.  In  addition,  all  subjects  were  given  a  battery  of  group 
administered  criterion  tests.  Their  scores  on  the  Scholastic  Aptitude  Test  (verbal  and 
math),  as  well  as  their  high  school  and  freshman  grade  point  averages,  were  recorded  from 
their  official  university  transcripts. 

Results 


Test-retest  reliability  coefficients  for  the  four  reaction  time  tests  ranged  from  .57  to 
*?i*  Test-retest  reliability  coefficients  for  the  three  inspection  time  tasks  ranged  from 
.25  to  .73,  with  the  highest  reliabilities  being  recorded  for  the  nonadaptive  tasks.  Test- 
retest  reliability  for  the  speed  of  processing  in  active  memory  (Mental  Counters)  test  was 
.59.  Split-half  reliabilities  were  generally  higher  than  those  for  test-retest.  Validity 
coefficients  were  mixed. 

Discussion  and  Conclusions 


The  following  conclusions  appear  to  be  warranted  from  the  data. 

1.  Reaction  time  (RT)  and  inspection  time  (IT)  tests  load  on  separate  factors,  and 
may  have  different  patterns  of  validity.  Certain  tests  of  each  construct  have  sufficient 
reliability  to  justify  further  development. 

2.  RT  tasks  of  all  types  are  highly  intercorrelated  and  related  similarly  to  a  variety 
of  criterion  variables.  Consequently,  it  may  not  be  necessary  to  have  a  wide  battery  of 
such  tasks— one  or  two  well-selected  tasks  are  all  that  is  needed.  Of  the  RT  tests 

administered  in  the  present  study,  the  Arrows  test  is  the  best  prospect  for  further 
research. 


^  tasks,  those  involving  horizontal  lines  in  a  nonadaptive  paradigm  are  by 
far  the  most  reliable.  Increasing  the  number  of  trials  from  50  to  75  would  result  in  a 


better  test.  A  nonadaptive,  75  item  horizontal  line  test  and  the  newer  Perceptual 
Organization  IT  test  both  deserve  further  evaluation. 

4.  The  Mental  Counters  Test  is  particularly  promising  in  that  it  loads  on  its  own 
factor  and  thus  appears  to  be  tapping  into  a  source  of  variance  different  from  RT  and  IT 
tasks. 


5.  The  continued  development  and  validation  of  measures  of  RT,  IT,  and  other 
variables  appears  to  hold  considerable  promise  for  measuring  aspects  of  ability  not  tapped 
by  conventional  psychometric  tests. 
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INTRODUCTION 


There  has  been  almost  no  increase  in  the  predictive  validity  of  traditional  psycho¬ 
metric  tests  since  they  were  first  put  into  use  more  than  60  years  ago  (Rundquist,  1969). 
The  development  of  economical  high  speed  microcomputers  and  recent  research  in 
cognitive  psychology  have,  however,  led  to  a  new  approach  of  considerable  promise: 
cognitive  assessment.  Cognitive  assessment  is  based  on  theories  of  information 
processing,  which  view  mental  ability  in  terms  of  a  number  of  underlying  component 
processes  (e.g.,  sensory  input,  mental  transformations  of  data,  storage,  etc.).  One 
attempts  to  determine  the  importance  of  various  component  mental  processes  to 
individual  differences  in  ability. 

One  branch  of  cognitive  assessment  emphasizes  mental  speed.  There  appear  to  be 
two  fundamentally  different  ways  of  timing  elementary  mental  processes  via  micro¬ 
computer:  one  based  on  reaction  time  (RT);  the  other  based  on  a  measure  of  information 
processing  (mental)  speed  called  inspection  time  (IT)  (Vernon,  1986). 

In  the  RT  tasks,  the  subject  is  asked  to  respond  as  quickly  as  possible  following  the 
presentation  of  a  stimulus.  The  nature  of  the  task  varies  from  simple  RT,  in  which  the 
subject  merely  responds  as  quickly  as  possible  following  the  onset  of  a  single  stimulus,  to 
complex  (i.e.,  choice)  RT,  in  which  decisional  processes  are  involved.  It  is  well  known 
that  multiple-choice  RT  increases  as  a  linear  function  of  the  logarithm  (to  base  2)  of  the 
number  of  choices,  a  phenomenon  known  as  Hick's  Law  (Hick,  1952;  Jensen,  1986).  The 
slope  of  this  function  is  generally  regarded  as  an  inverse  measure  of  speed  of  processing. 
A  number  of  studies  have  found  a  modest  (median  about  .4)  but  consistent  correlation 
between  a  variety  of  RT  tasks  and  traditional  psychometric  intelligence  tests  (Carlson  <5c 
Jensen,  1982;  Carroll,  1980;  Vernon  Sc  Jensen,  1984).  Reported  reliabilities  for  various  RT 
tasks  vary,  but  fall  within  very  acceptable  limits.  Jensen  (1982)  reports  that  the  typical 
test-retest  reliabilities  from  his  lab  run  about  .75.  Krause  and  Bittner  (1982)  report  test- 
retest  coefficients  of  .58,  .51,  and  .80  for  a  one,  two,  and  four  RT  task,  respectively. 
Other  studies  report  similar  correlations,  with  a  range  of  coefficients  from  the  low  .5s  to 
the  low  .8s  and  median  in  the  mid  .7s  (e.g.,  see  Bittner,  Carter,  Kennedy,  Harbeson,  Sc 
Krause,  1984;  Carroll,  1980;  Harbeson,  Kennedy,  Krause,  Sc  Bittner,  1982;  Kyllonen,  1986; 
Rose  Sc  Fernandez,  1977).  Split-half  reliabilities  for  simple  and  choice  RT  tasks  typically 
run  above  .90  (see  Vernon,  1983). 

*n  tasks,  subjects  are  presented  with  increasingly  rapid  stimulus  presentations. 
Mental  (cognitive)  speed  is  evaluated  in  terms  of  a  subject's  ability  to  respond  accurately 
to  such  presentations  rather  than  in  terms  of  speed  of  response  in  pressing  an  appropriate 
key  as  in  RT  tasks  (Vernon,  1986;  Vickers,  Nettelbeck  Sc  Wilson,  1972;  Vickers  Sc  Smith, 
1986).  For  example,  two  lines  of  unequal  length  may  be  presented  at  varying  speeds. 
Following  termination  of  the  line  stimuli  there  immediately  follows  a  visual  backward 
mask,  which  limits  stimulus  processing  (Felsten  Sc  Wasserman,  1980;  Saccuzzo,  Larson,  Sc 
Rimland,  1986).  IT  is  based  on  the  fastest  speed  at  which  a  subject  obtains  a  criterion 
degree  of  correct  identifications  of  thee  longer  line.  A  large  body  of  literature  has 
documented  a  modest  but  clear-cut  relationship  between  intelligence  and  both  RT  and  IT 
time  measures  of  cognitive  speed,  supporting  the  potential  of  these  relatively  new 
measures  in  cognitive  assessment  (Longstreth,  Walsh,  Alcorn,  Szeszulski,  Sc  Manis,  1986- 
Nettelbeck,  Edwards,  Sc  Vreugdenhil,  1986;  Saccuzzo,  Larson,  <5c  Rimland,  1986). 

Based  on  a  number  of  small  sample  studies  conducted  at  their  lab,  Brand  and  Deary 
(1982)  report  that  the  typical  reliability  coefficient  for  inspection  time  is  .8,  a  figure  in 
accord  with  a  report  by  Vernon  (1983),  who  also  reports  a  test-retest  reliability 
coefficient  of  .8  for  an  IT  task.  As  Nettelbeck  (1983)  noted,  however,  to  date  there  has 
yet  to  be  a  reliability  study  that  has  used  a  reasonable  large  sample.  In  a  review  of  the 
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literature,  Nettelbeck  (1982,  1983)  found  36  instances  involving  non-retarded  adult 
samples  (usually  university  students)  that  ranged  in  size  from  10  to  56  subjects;  reliability 
coefficients  for  IT  ranged  from  .25  to  .92,  with  an  average  of  .65.  As  Nettelbeck  points 
out,  however,  the  vast  majority  of  these  coefficients  are  based  on  repeated  measures 
taken  within  a  single  session,  or  at  most,  within  a  day  or  two.  On  the  whole,  correlations 
have  been  least  impressive  for  larger  sample  sizes  and  longer  test-retest  intervals. 

In  summarizing  the  limited  data  on  RT  and  IT  tasks,  two  major  conclusions  emerge. 
First,  tests  involving  simple  and  choice  reaction  time  are  highly  reliable  for  indices  based 
on  response  latencies.  Median  reported  test-retest  coefficients  run  in  the  mid  .7s,  while 
internal  consistency  coefficients  run  in  the  high  ,9s.  Second,  on  the  whole,  the  reliability 
for  IT  appears  to  be  lower  than  for  RT,  but  within  acceptable  limits.  However,  more  work 
on  larger  samp'es  is  needed. 

The  primary  purpose  of  the  present  study  was  to  provide  test-retest  reliability  data 
on  a  battery  of  microcomputerized  cognitive  speed  tasks  developed  by  researchers  at  the 
Naval  Personnel  Research  and  Development  Center  (NAVPERSRANDCEN),  San  Diego. 
The  battery  includes  a  number  of  standard  RT  and  IT  tasks,  adapted  for  microcomputer 
presentations,  as  well  as  a  number  of  experimental  complex  RT  tasks  (the  Arrows  Test 
and  a  RT  paradigm  not  involving  movement  time),  an  experimental  IT  task  (a  perceptual 
organization  task),  and  a  measure  of  storage  and  processing  speed  in  active  memory  called 
the  Mental  Counters  Test  (MCT)  developed  by  Larson  (in  preparation).  In  addition,  a 
number  of  questions  were  addressed.  In  particular,  this  study  attempted  to  evaluate:  (1) 
the  relative  reliabilities  of  different  approaches  to  evaluating  IT  (i.e.,  adaptive  versus 
nonadaptive),  (2)  the  effect  of  increasing  task  complexity  on  subject  performance,  and  (3) 
the  validity  of  RT  and  IT  measures. 

All  of  the  tasks  chosen  for  study  were  nonverbal,  knowledge  free,  and  involved 
minimal  response  requirements  (i.e.,  all  a  subject  was  required  to  do  was  press  an 
appropriate  key  on  the  microcomputer  keyboard).  They  varied  primarily  in  terms  of 
complexity  and  whether  the  task  emphasized  response  speed  or  response  accuracy. 


METHOD 


Subjects 

The  subjects  were  104  volunteer  San  Diego  State  University  students  from  an 
introductory  course  in  psychology.  They  ranged  in  age  from  18  to  35-years-old  (M  =  21.39, 
SD  -  3.72).  The  majority  of  subjects  were  19-years-old  (N  =  29),  20-years-old  (N  =  24),  or 
21-years-old  (N  =  13).  There  were  49  males  and  55  females.  Eighty-four  were  Caucasian, 
7  Hispanic,  2  Black,  6  Asian,  and  5  "other."  The  sample  was  representative  of  San  Diego 
State  University  students. 

Procedure 


Each  subject  was  tested  on  a  battery  of  microcomputerized  tests.  Presentation  of 
these  tests  was  completely  randomized  for  each  subject  according  to  a  prearranged 
random  sequence.  Three  types  of  tasks  were  used:  (1)  IT,  (2)  RT,  and  (3)  Mental 
Counters. 

1.  Inspection  Time  Tasks.  There  were  three  IT  tasks:  (a)  Inspection  Time  Test- 
Adaptive  (IT- Adaptive),  (b)  Inspection  Time  Test-Nonadaptive  (IT-Nonadaptive),  and  (c) 
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the  Perceptual  Organization  Test  (PO  Test).  For  all  IT  tasks,  a  visual  stimulus  was  briefly 
presented.  Immediately  following  stimulus  termination,  a  backward  visual  noise  mask  was 
presented.  The  mask  is  known  to  limit  the  duration  of  the  sensory  signal  delivered  to  the 
central  nervous  system  (CNS)  (Felsten  &  Wasserman,  1980).  The  subject's  task  was  to 
make  a  forced-choice  discrimination  by  pressing  one  of  two  buttons  on  the  microcomputer 
keyboard.  Each  of  these  three  IT  tasks  is  briefly  described  below. 

a.  Inspection  Time  Test-Adaptive  (IT-Adaptive) 

The  IT-Adaptive  was  an  IBM-PC  presented  version  of  the  task  used  by 
Larson  and  Rimland  (1984).  In  this  task,  subjects  are  briefly  shown  two  lines  of  unequal 
length,  which  are  presented  in  the  center  of  the  cathode  ray  tube  (CRT)  screen.  The 
subject's  task  is  to  identify  the  longer  line  by  pressing  an  appropriate  key  on  the  keyboard. 
The  longer  line  is  randomly  presented  either  to  the  right  or  left  of  central  fixation  on  any 
given  trial.  Thus,  the  task  involves  a  forced-choice  visual  discrimination.  To  control  for 
individual  differences  in  visual  persistence  and  for  the  duration  of  the  sensory  signal 
delivered  to  the  CNS,  presentation  of  the  lines  is  immediately  followed  by  a  visual  noise 
mask  consisting  of  a  solid  parallel  line  that  completely  superimposes  the  target  line 
stimuli.  The  duration  of  the  target  line  stimuli  was  varied,  depending  on  subject 
performance.  Presentations  began  with  an  initial  display  duration  of  317.3  msec.  In  the 
display  duration  algorithm,  if  a  subject  made  three  consecutive  correct  responses,  he  or 
she  was  allowed  to  attempt  the  next  higher  level  of  difficulty.  If,  however,  the  subject 
made  an  error,  the  test  decreased  in  difficulty  (i.e.,  the  stimulus  duration  was  increased). 
If  the  subject  made  three  consecutive  responses  at  the  briefest  display  duration  possible 
on  the  CRT  screen  (16.7  msec.),  the  testing  was  terminated  and  the  subject  was  assigned  a 
score  of  "0."  Otherwise,  testing  continued  until  the  subject  made  a  total  of  three  errors; 
the  subject's  score  was  then  based  on  the  final  display  speed,  with  lower  scores  denoting 
better  performance.  The  test  was  administered  twice;  once  with  vertical  lines  as  test 
items  (line  lengths  of  20.6  mm.  and  14.3  mm.),  and  once  with  horizontal  lines  (17.5  mm. 
and  14.3  mm.).  Although  the  line  lengths  differed  between  orientations,  a  smaller 
disparity  was  employed  in  the  horizontal  condition  due  to  results  from  pilot  testing,  which 
indicated  that  the  discrimination  was  easier  for  horizontal  lines. 

b.  Inspection  Time  Test-Nonadaptive  (IT-Nonadaptive) 

The  IT-Nonadaptive  was  identical  to  the  above  procedure  with  the  exception 
that  the  sequence  of  stimulus  durations  used  was  fixed  in  advance,  rather  than  dependent 
on  subject  performance.  Subjects  began  with  10  trials  of  either  horizontal  or  vertical 
lines  for  each  of  five  display  speeds:  16.7,  33.4,  66.8,  100.2,  and  150.3.  Order  of 
presentation  of  display  speeds  was  completely  randomized  for  each  trial,  with  the 
exception  that  no  more  than  10  trials  would  occur  for  any  given  stimulus  duration  within  a 
total  of  50  trials.  After  the  first  50  trials,  the  entire  procedure  was  repeated  using 
horizontal  lines  if  the  subject  began  with  vertical  lines  and  vice  versa.  Thus,  subjects  had 
a  total  of  100  trials,  20  at  each  of  five  different  stimulus  durations. 

c.  Perceptual  Organization  Test  (PO  Test) 

The  PO  Test  is  a  variation  of  IT  in  which  the  discrimination  must  be  made 
between  briefly  displayed  patterns  of  dots  organized  into  either  rows  or  columns.  The 
rationale  for  the  PO  Test  is  based  on  a  series  of  studies  that  revealed  a  relationship 
between  tested  intelligence  and  performance  on  a  tachistoscopic  version  of  the  test  (De 
Soto  &  Leibowitz,  1956;  Krech  &  Calvin,  1953).  As  with  the  IT-Nonadaptive  Test, 
stimulus  are  randomly  presented  at  any  one  of  five  display  speeds:  16.7,  50.1,  83.5,  167, 


3 


and  334  msec.  Immediately  following  termination  of  the  pattern  display,  a  spatially 
overlapping,  nonpatterned  cluster  of  dots  was  presented,  providing  a  visual  backward  mask 
that  limited  the  duration  of  the  sensory  signal  delivered  to  the  central  nervous  system. 
There  were  15  trials  per  display  speed,  with  order  of  presentation  completely  randomized. 
At  the  beginning  of  each  trial,  subjects  were  instructed  to  attend  to  a  fixation  point  in  the 
center  of  the  screen.  After  one  second,  a  dot  pattern,  approximately  16  mm.  square,  was 
presented  next  to  fixation,  offset  in  the  direction  of  one  of  the  four  screen  corners,  but 
overlapping  with  fixation  so  that  one  of  the  corners  of  the  stimulus  pattern  was  anchored 
to  screen  center.  The  pattern  was  made  of  dots  spaced  approximately  4  mm.  apart. 
Subjects  were  instructed  to  respond  by  pressing  an  appropriate  key,  depending  on  whether 
they  had  perceived  an  upright  (columns)  or  sideways  (rows)  pattern. 

2.  Reaction  Time  Paradigms.  Four  RT  paradigms  were  used:  (a)  Hick  Paradigm 
with  movement  time  for  1,  3,  and  5  choices;  (b)  Hick  Paradigm  without  movement  time 
for  1,  3,  and  5  choices:  (c)  the  Arrows  Test;  and  (d)  a  two  choice  reaction  time  test  for 
central,  right,  and  left  visual  field  presentations.  Each  of  these  is  briefly  described 
below. 


a*  Hick  Paradigm  with  Movement  Time  for  1,  3,  and  5  Choices 

This  paradigm  is  essentially  the  same  as  that  employed  in  previous  research 
(see  Larson  &  Rimland,  1984;  Saccuzzo  et  al.,  1986).  A  horizontal  arrangement  of  lights 
was  presented  at  the  bottom  of  the  CRT  screen,  and  the  top  row  of  keys  on  the  keyboard 
(one  key  paired  with  each  stimulus  light)  was  used  for  responding.  All  subjects  were 
presented  with  1-,  3-,  and  5-choice  conditions,  with  order  of  presentation  completely 
randomized.  Open  squares  on  the  CRT  screen  were  used  as  stimulus  lights,  and  subjects 
were  instructed  to  respond  by  first  pressing  the  space  bar  and  then  pressing  the 
appropriate  key  as  quickly  as  possible  after  a  square  became  illuminated.  There  were  21 
trials  at  each  condition.  At  the  beginning  of  each  trial,  subjects  rested  the  forefinger  of 
their  dominant  hand  on  the  space  bar  at  the  bottom  of  the  keyboard.  After  a  random 
period  of  time  (1.5  to  2.5  seconds),  one  of  the  stimulus  squares  was  illuminated.  Reaction 
time  was  the  number  of  msec,  between  the  onset  of  the  stimulus  (i.e.,  when  one  of  the 
stimulus  squares  was  illuminated)  and  the  instant  the  subject  pressed  the  space  bar. 
Movement  time  was  the  number  of  msec,  between  pressing  the  space  bar  and  striking  a 
response  key.  If  a  reaction  time  greater  than  2  seconds  was  recorded,  the  trial  was 
discarded  and  a  new  one  presented  to  maintain  a  total  of  21  trials  per  condition.  A  count 
was  kept  of  discarded  trials. 

b.  The  Arrows  Test 


In  the  Arrows  Test  (Larson,  1985),  subjects  were  instructed  to  fixate  on  two 
small  circles  presented  side  by  side  in  the  center  on  the  CRT  screen.  For  each  trial,  one 
of  the  circles  was  replaced  by  an  arrow,  and,  depending  on  its  direction  and  position,  the 
subject  responded  by  pressing  either  a  right  or  left  key  on  the  microcomputer  keyboard. 
If  the  arrow  pointed  down,  its  position  indicated  the  appropriate  response.  For  example, 
if  a  downward  arrow  replaced  the  right  circle,  the  right  key  was  pressed.  If  a  downward 
pointing  arrow  replaced  the  left  circle,  then  the  left  key  was  pressed.  If  an  arrow 
pointing  sideways  (i.e.,  right  or  left)  was  present,  then  direction  became  the  relevant  cue, 
while  position  became  a  distractor.  If  the  arrow  pointed  right,  the  right  key  was  pressed. 
If  it  pointed  left,  the  task  was  to  press  the  left  key  as  soon  as  possible.  RTs  greater  than 
2  seconds  were  discarded  and  new  items  presented  to  maintain  a  constant  number  of  trials 
per  subject.  A  count  was  kept  of  discarded  trials.  The  position  and  direction  of  the  arrow 
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were  varied  randomly.  The  test  involved  82  trials;  41  with  downward  arrows  and  41  with 
right-left  arrows. 

c*  Two  Choice  Reaction  Time  Test  for  Central,  Right,  and  Left  Visual  Field 

Presentations 


In  the  two  choice  reaction  time  test  for  central,  right,  and  left  presenta¬ 
tions,  subjects  were  randomly  presented  with  either  of  the  following  two  patterns: 
"X:*:X"  or  "X:+:X."  The  patterns  were  presented  at  three  screen  locations.  On  50 
percent  of  trials,  the  pattern  appeared  at  a  fixation  point  in  the  middle  of  the  screen,  so 
that  the  central  element  of  the  pattern  ("*"  or  "+")  replaced  the  fixation  point.  For  the 
remaining  trials,  the  patterns  were  presented  at  either  the  far  left  or  far  right  of  the  CRT 
screen.  Thus,  detection  required  a  visual  shift  from  the  central  fixation  point  to  the  site 
of  the  item.  If  a  RT  greater  than  2  seconds  for  any  condition  was  recorded,  the  trial  was 
discarded  and  a  new  one  presented  to  maintain  a  constant  number  of  trials  per  subject, 
per  condition.  A  count  was  kept  of  discarded  responses.  The  test  involved  80  trials;  40  at 
fixation  (screen  center),  20  at  screen  right,  and  20  at  screen  left.  By  subtracting  RT  at 
center  from  RT  at  periphery,  the  time  required  to  make  a  visual  shift  was  isolated,  since 
all  other  processes  involved  in  visual  encoding  and  response  selection  and  execution  are 
presumed  to  be  equal  across  screen  locations. 

3.  Mental  Counters  Test  (MCT).  In  the  MCT  (Larson,  in  preparation),  subjects  must 
keep  track  of  the  values  of  three  independent  "counters."  The  values  change  rapidly  and 
in  random  order.  The  difficulty  of  the  task  comes  from  having  to  simultaneously  hold  the 
three  counter  values  in  memory,  rapidly  update  those  values  as  necessary  based  on  a 
simple  arithmetic  calculation,  and  store  the  new  values.  If  counter  updating  is  performed 
too  slowly,  the  adjustments  themselves  must  be  remembered,  even  as  new  calculations  are 
required.  Individuals  who  must  store  too  many  adjustments  because  of  slow  execution  will 
eventually  experience  a  "breakdown"  as  capacity  is  exceeded. 


The  counters  themselves  are  represented  as  lines  on  the  video  monitor  (three  side  by 
side  horizontal  dashes  in  the  center  of  the  screen).  The  initial  counter  values  are  zero. 
When  a  target  (a  small  box)  appears  above  a  dash,  the  corresponding  counter  must  be 
adjusted  by  adding  "1."  When  the  target  appears  below  one  of  the  three  dashes,  the 
corresponding  counter  must  be  adjusted  by  subtraction  "-1."  The  test  items  vary  both  in 
the  number  of  targets  (i.e.,  number  of  counter  adjustments)  and  in  rate  of  presentation. 
There  were  two  levels  of  counter  adjustments  (five  and  seven)  and  two  levels  of  rate  of 
presentation  (fast  and  slow).  The  actual  test  involved  a  total  of  40  trials.  On  20  trials,  5 
targets  were  presented.  Seven  targets  were  presented  on  the  remaining  20  trials.  On  20 
trials,  targets  were  presented  at  the  rate  of  one  every  .25  of  a  second.  On  the  remaining 
20  trials,  targets  were  presented  at  the  rate  of  one  every  .83  of  a  second.  Number  of 
targets  and  rate  of  presentation  were  completely  counterbalanced. 

Table  1  presents  a  summary  of  the  microcomputerized  tests. 

All  microcomputerized  tests  were  presented  on  IBM  PC/XT  microcomputers  with 
color  monitors  and  standard  keyboards.  No  special  add-ons  were  used  other  than  color 
labeling  of  response  keys.  Total  testing  time  for  the  entire  battery  of  tests  was 
approximately  1  hour  and  20  minutes. 


5 


Table  1 


Summary  of  Microcomputerized  Tests  and  Acronyms 


Test 

Acronym  or  Brief  Name 

Inspection  Time  Test-Adaptive 

IT-Adaptive 

Inspection  Time  Test-Nonadaptive 

IT  Non-Adaptive 

Perceptual  Organization  Test 

PO  Test 

Hick  Paradigm  with  Movement  for 

1  Choice 

Hick  Move  1 

3  Choices 

Hick  Move  3 

5  Choices 

Hick  Move  5 

Hick  Paradigm  Without  Movement  for 

1  Choice 

Hick  No  Move  1 

3  Choices 

Hick  No  Move  3 

5  Choices 

Hick  No  Move  5 

Arrows  Test 

Arrows 

Two  Choice  Reaction  Time  Test  for 

Central 

Choice  RT 

Right 

Left 

Right  Choice  RT 

Left  Choice  RT 

Mental  Counters  Test 

MCT 

Since  the  primary  purpose  of  this  study  was  to  evaluate  the  test-retest  reliabilities 
for  the  microcomputerized  tests,  subjects  were  invited  to  return  for  a  retesting.  Only 
those  subjects  who  could  be  retested  within  a  period  of  not  less  than  24-hours,  but  not 

more  than  10  days,  were  retested.  Seventy-four  subjects  met  this  criterion  and  were 
retested. 

Group  Criterion  Tests 

During  the  course  of  the  study,  each  of  the  original  104  subjects  were  given  the 
following  five  group  administered  criterion  tests. 

^  test  (Identical  Picture  Test)  of  the  kit  of  Factor-Referenced  Cognitive 

Tests  (Ekstrom,  French,  <5c  Harman,  1976).  The  P3  test  requires  subjects  to  match  simple 
scanning  **  eVa*Uates  a  sut)ject?s  speed  in  carrying  out  simple  tasks  involving  perceptual 

2*  The  SI  test  (Card  Rotations  Test)  of  the  kit  of  Factor-Referenced  Cognitive 

Tests.  A  measure  of  spatial  orientation — the  ability  to  perceive  spatial  patterns  or 

maintain  orientation  with  respect  to  objects  in  space.  The  SI  tests  requires  mental 
rotation  of  figures. 
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3-  The  VZ3  (Surface  Development  Test)  of  the  kit  of  Factor-Referenced  Cognitive 

Tests.  This  is  the  highest  loading  test  of  the  Visualization  Factor,  which  is  defined  as  the 

ability  to  manipulate  or  transform  the  images  of  spatial  patterns  into  other  arrangements. 

.  ..  Cognitive  Laterality  Battery  (CLB),  developed  by  Gordon  (1983)  to  evaluate 

individual  differences  in  hemispheric  asymmetries.  The  CLB  is  a  slide  projected  battery 
of  eight  individual  subtests.  Presentation  of  the  battery  is  driven  by  cues  embedded  in 
cassette  tapes,  which  contain  instructions  and  auditory  test  items.  Of  the  eight  subtests, 
four  are  nonverbal  and  purportedly  related  to  right  hemisphere  functioning;  the  other  four 
are  related  to  left  hemisphere  functioning.  An  overall  Right  Hemisphere  (RH)  score  is  the 
average  of  the  Z-scores  for  the  four  right-hemisphere-related  subtests;  an  overall  Left 
Hemisphere  (LH)  score  is  the  average  of  the  Z-scores  for  the  four  left-hemisphere-related 

IgSKS# 


5-  The  Raven  Progressive  Matrices  Test.  Advanced 
time  limit). 


,  group  administered  (40  minute 


In  addition,  each  subjects'  score  on  the  Scholastic  aptitude  Test,  Verbal  (SATV)  and 
Math  (SATM),  as  well  as  high  school  grade  point  average  (HSGPA)  and  freshman  grade 
point  average  (FRGPA)  were  recorded  from  their  official  transcripts.  Table  2  provides  a 
summary  of  the  criterion  measures. 


Table  2 

Summary  of  Criterion  Measures  and  Acronyms 


Variable 

Measure  of 

Acronym 

Identical  Picture  Test 

Perceptual  Speed 

P3 

Card  Rotations  Test 

Spatial  Orientation 

SI 

Surface  Development  Test 

Visualization 

VZ3 

Gordon  Cognitive  Laterality  Battery 

Overall  Right  Hemisphere 

Right  Hemisphere  Functioning 

GRH 

Gordon  Cognitive  Laterality  Battery 

Overall  Left  Hemisphere 

Left  Cerebral  Hemisphere 

GLH 

Raven  Progressive  Matrices  Test 

Intelligence 

Raven 

Scholastic  Aptitude  Test,  Verbal 

Scholastic  Aptitude  (Verbal) 

SATV 

Scholastic  Aptitude  Test,  Math 

Scholastic  Aptitude  (Mathematics) 

SATM 

High  School  Grade  Point  Average 

School  Achievement 

HSGPA 

Freshman  Grade  Point  Average 

School  Achievement 

FRGPA 

RESULTS 

Table  3  presents  the  means  and  standard  deviations  for  each  of  the  criterion  variables 
as  well  as  their  intercorrelations.  In  examining  these  correlations,  it  is  notable  that  the 
Raven  Progressive  Matrices  Test  correlated  well  with  the  SI,  VZ3,  and  Gordon  Right 
Hemisphere  Tests.  The  only  significant  correlate  of  the  SATV  was  the  SATM  (r  =  28)  § 
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Table  3 


Means,  Standard  Deviations,  and  Intercorrelation 
Among  Criterion  Variables 


P3 

SI 

VZ3 

GRH 

GLH 

Raven 

SATV 

SATM 

HSGPA 

FRGPA 

P3 

1.0 

.35** 

.39** 

.26* 

.18 

.25* 

1 

• 

o 

-.05 

.12 

1 

• 

o 

SI 

— 

1.0 

.37** 

.07 

.05 

.32* 

.20 

.28* 

.45** 

.18 

VZ3 

— 

— 

1.0 

-.00 

.54** 

.16 

.41** 

.24* 

.18 

GRH 

— 

— 

— 

1.0 

.12 

.36** 

.03 

-.02 

-.02 

.12 

GLH 

— 

— 

— 

— 

1.0 

.05 

.15 

.14 

-.20 

-.09 

Raven 

— 

— 

— 

— 

— 

1.0 

.18 

.47** 

.11 

.01 

SATV 

— 

— 

— 

— 

— 

— 

1.0 

.28* 

.07 

-.02 

SATM 

— 

— 

— 

— 

— 

— 

— 

1.0 

00 

o 

• 

1 

.10 

HSGPA 

— 

— 

— 

— 

— 

— 

— 

— 

1.0 

.54** 

FRGPA 

— 

— 

— 

— 

— 

— 

— 

— 

— 

1.0 

Mean 

80 

103 

30 

o 

• 

1 

.18 

22 

438 

499 

2.8 

2.2 

SD 

12 

33 

15 

.96 

.69 

5.45 

76 

89 

.65 

00 

*£  <  .05. 
**£<  .01. 


In  analyzing  the  data  for  the  computerized  tests,  a  number  of  preliminary  analyses 
were  conducted  to  determine  the  best  or  most  representative  indices  for  each.  For  the 
adaptive  IT  test,  results  from  the  vertical  and  horizontal  versions  were  converted  to  7,- 
scores  and  averaged  to  form  a  composite  score,  labeled  IT-Adaptive.  For  the  IT- 
Nonadaptive  test,  test-retest  reliabilities  were  calculated  separately  for  each  of  the  five 
stimulus  durations  for  the  vertical  and  horizontal  versions.  In  addition,  a  weighting  scheme 
was  explored  in  which  points  were  assigned  for  correct  responses  to  an  item  as  a  linear 
unction  of  that  item's  display  duration.  The  two  weighted  scores  (one  for  the  vertical 
version  and  one  for  the  horizontal  version)  were  converted  to  Z-scores  and  averaged  to 
form  a  composite.  Finally,  a  simple  measure  based  on  the  total  correct,  also  based  on 
average  Z-scores  for  the  vertical  and  horizontal  tests,  was  evaluated.  Table  4  presents  the 
various  test-retest  reliabilities  for  each  of  these  measures.  The  simple  measure  based  on 
*°tal  C0I7eCt  was  found  t0  be  of  higher  reliability  than  either  weighted  score  and,  labeled 
IT-Nonadaptive,  is  used  throughout  the  remainder  of  the  analysis.  It  should  be  noted, 
however,  that  the  task  involving  horizontal  lines  (rXx  =  .66)  was  clearly  more  stable  than 
the  task  based  on  vertical  lines  (rxx  .41). 


Table  4 

Test-Retest  Reliabilities  for  Various  Indices  of  IT-Nonadaptive 


Horizontal  Lines 

16.7  msec,  duration 
33.4  msec,  duration 

66.8  msec,  duration 

100.2  msec,  duration 

150.3  msec,  duration 

Total  correct  based  on  horizontal  lines 
Vertical  Lines 

16.7  msec,  duration 
33.4  msec,  duration 

66.8  msec,  duration 

100.2  msec,  duration 

150.3  msec,  duration 

Total  correct  based  on  vertical  lines 

Weighted  score  based  on  horizontal  lines 

Weighted  score  based  on  vertical  lines 

Weighted  score  based  on  both  horizontal  and  vertical  lines 

Total  correct  based  on  both  horizontal  and  vertical  lines 

aN  =  72. 

*£  <  .05. 

**£<  .01. 


Reliability  Coefficient3 


.27* 

.20* 

.43** 

.64** 

.59** 

.66** 


.01 

.11 

.12 

.29* 

.57** 

.41** 

.49** 

.22* 

.39** 

.55** 
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For  the  perceptual  organization  test,  test-retest  reliabilities  were  determined  for  the 
total  correct  at  each  of  the  five  stimulus  durations.  These  were,  from  the  lowest  to  the 
highest  durations,  respectively,  .41,  .54,  .54,  .56,  .59  (based  on  an  N  of  74  valid  cases).  A 
derived  weighted  score,  which  gave  subjects  more  points  for  correct  responses  on  more 
difficult  items,  had  a  test-retest  coefficient  of  .56.  The  best  index,  however,  was  based 
on  the  total  correct  across  the  entire  test  (rxx  =  .73)  and  this  index  is  used  throughout  the 
remainder  of  the  analyses. 

All  reaction  time  indices  were  based  on  the  median  response  latencies.  For  the 
Arrows  Test,  two  such  latencies  were  included,  Arrows  Down  (median  latency  for  arrows 
that  pointed  downward)  and  Arrows  Side  (median  latency  for  arrows  that  pointed  right  or 
left).  For  the  two  choice  reaction  time  test,  three  indices  were  calculated:  median 
latency  for  central  presentation  (rxx  =  .74),  median  latency  for  left  presentations 
(rxx  =  .67),  and  the  median  latency  for  right  presentations  (rxx  =  .58).  Because  of  the 
extremely  high  intercorrelations  among  these  three  indices,  however,  only  the  central 
presentations  measure,  labeled  simply  "Choice  RT,"  is  included  in  the  subsequent  analyses. 

Table  5  shows  the  test-retest  reliabilities  for  the  various  indices  of  the  MCT.  The 
index  based  on  the  total  correct  for  all  trials  provided  one  of  the  best  measures,  and  is 
used  throughout  the  remainder  of  the  analysis  and  is  labeled  MCT. 


Table  5 

Test-Retest  Reliabilities  for  the  Mental  Counters  Test 


Reliability  Coefficients3 

Slow  speed,  seven  indicators 

.50* 

Fast  speed,  seven  indicators 

.49* 

Slow  speed,  five  indicators 

.55* 

Fast  speed,  five  indicators 

.41* 

Seven  indicators,  total  correct 

.47* 

Five  scores,  total  correct 

.62* 

Total  correct  based  on  all  trials 

.59* 

aN  =  74. 

*£  <  .001. 


Table  6  provides  an  overall  summary  of  the  means,  standard  deviations,  and  test- 
retest  reliability  coefficients  for  each  of  the  major  indices.  Examination  of  Table  6 
reveals  that,  with  the  exception  of  IT-Adaptive,  all  of  the  test-retest  reliabilities  were 
greater  than  .55.  Overall,  reliabilities  were  greater  for  reaction  time  tasks,  as  compared 
to  the  inspection  time  tasks  and  the  MCT. 
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Table  6 


Means,  Standard  Deviations,  and  Reliability  Coefficients 
for  Computerized  Tests 


Test 

Ml 

M2 

SD1 

SD2 

r  H 

rxx 

Inspection  Time 

Adaptive 

-0.03 

-0.02 

0.74 

0.81 

.25* 

Nonadaptive 

0.33 

0.04 

0.77 

0.50 

.55* 

Perceptual  organization 

65.74 

66.88 

6.00 

7.42 

.73* 

Hick  Paradigm 

RT  with  movement 

1  choice 

310.53 

294.16 

63.82 

62.55 

.62* 

3  choices 

395.73 

373.51 

79.90 

73.16 

.72* 

5  choices 

430.44 

419.39 

92.28 

100.32 

.75* 

RT  without  movement 

1  choice 

271.29 

261.64 

61.03 

43.18 

.59* 

3  choices 

322.34 

321.07 

61.46 

75.23 

.65* 

5  choices 

352.58 

359.01 

74.96 

92.62 

.57* 

Arrows  Test 

Down  median 

487.19 

440.04 

77.10 

75.46 

.73* 

Side  median 

551.14 

500.68 

101.26 

98.33 

.81* 

Choice  RT 

550.70 

530.43 

65.41 

73.67 

.74* 

Mental  Counters  Test 

26.81 

29.74 

7.16 

6.18 

.59* 

a(N  =  74). 
<  .01. 


Split-half  reliability  coefficients  are  presented  in  Table  7.  The  table  includes 
coefficients  based  on  the  total  sample  as  well  as  on  only  that  subset  of  the  total  sample 
that  was  retested.  Because  of  the  method  used  to  calculate  IT-Adaptive,  it  was  not 
possible  to  compute  split-half  coefficients  for  this  test.  Similarly,  it  was  not  possible  to 
calculate  split-half  coefficients  for  composite  scores.  Examination  of  Table  7  reveals 
the  split-half  coefficients  are  generally  higher  (except  for  IT-Nonadaptive,  vertical 
lines)  then  the  test-retest  coefficients  for  all  variables  for  which  the  split-half  reliability 
coefficient  was  determined. 

Table  8  provides  a  summary  of  the  tests  of  significance  for  the  difference  between 
the  means  of  first  and  second  testing  (first  session  minus  second  session).  As  the  table 
shows,  significant  practice  effects  were  found  for  mental  counters  and  several  of  the  RT 
indices,  but  not  for  IT.  With  the  exception  of  5  choice  RT  without  movement,  all  practice 
effects  indicate  improvement  in  performance. 
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Table  7 


Split-Half  Reliability  Coefficients 


Test 

Full  Sample3 

First  Session*3 

Second  Session*3 

IT-Nonadaptive 
(Horizontal  lines) 

.77 

.73 

.77 

IT-Nonadaptive 
(Vertical  lines) 

.36 

.26 

.54 

Perceptual  Organization  Test 

.64 

.67 

.79 

Arrows  Test 

.95 

.96 

.95 

Mental  Counters  Test 

.77 

.79 

.68 

Hick  with  movement 

1  choice 

.71 

.75 

.71 

3  choices 

.73 

.72 

.74 

5  choices 

.72 

.75 

.86 

Hick  without  movement 

1  choice 

.78 

.74 

.74 

3  choices 

.87 

.84 

.85 

5  choices 

.75 

.73 

.78 

aN  =  104. 

^Includes  only  subjects  who  were  tested  both  times,  N  =  74. 
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Table  8 


Summary  of  t-tests  for  First  and  Second  Testings 


Difference 
Between  Means 

SD 

T-Value 

Df 

2-Tail  Prob. 

Inspection  Time3 

Adaptive 

Horizontal 

-0.83 

4.49 

-1.55 

69 

NS 

Vertical 

-0.35 

4.11 

-0.67 

70 

NS 

Nonadaptive 

Horizontal 

0.39 

5.07 

0.64 

69 

NS 

Vertical 

-0.24 

7.26 

-.26 

70 

NS 

Perceptual  organization 

-1.14 

4.95 

-1.95 

71 

NS 

Hick  Paradigm 

RT  with  movement 

1  choice 

16.08 

54.94 

2.50 

72 

£  <  .01 

3  choices 

22.54 

57.58 

3.37 

73 

£  <  *001 

5  choices 

11.06 

68.90 

1.36 

71 

NS 

RT  without  movement 

1  choice 

9.64 

49.61 

1.66 

72 

NS 

3  choices 

1.27 

59.01 

0.19 

73 

NS 

5  choices 

-6.43 

79.86 

-0.69 

73 

NS 

Arrows  Test 

Down  median 

47.15 

55.84 

7.26 

73 

£  <  .001 

Side  median 

50.46 

60.88 

7.13 

73 

£  <  .001 

Choice  RT 

20.27 

51.75 

3.37 

73 

£  <  .001 

Mental  Counters  Test 

-2.93 

6.10 

-4.14 

73 

£  <  .001 

All  values  for  inspection  time  are  based  on  raw  scores. 


The  following  analyses  are  based  on  the  full  sample  (N  =  104).  Table  9  presents  the 
intercorrelations  and  unrotated  factor  loadings  for  each  of  the  major  indices  for  the 

omputerized  battery.  The  table  shows  a  high  degree  of  intercorrelation  among  the 
reaction  time  variables.  6 


Table  10  reveals  the  varimax  rotated  factor  matrix  for  the  variables  listed  in  Table  9. 
Three  factors  emerged.  These  were  RT,  on  which  all  the  RT  tests  loaded:  Visual 
Processing  Speed,  on  which  the  three  IT  tasks  loaded,  and  Mental  Counters,  on  which  the 
highest  loading  was  the  MCT.  Salient  factor  loadings  are  underlined. 
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Table  9 


Intercorrelations  and  Unrotated  Factor  Loading 


For  Computerized  Tests 

1 

2  3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Loading3 

I. 

IT- Adaptive 

1.0 

-.30**  -.27** 

.17 

.27** 

.24** 

.22* 

.25** 

.23* 

.31** 

.32** 

.33** 

-.14 

.43 

2. 

IT-Nonadaptive 

— 

1.0  .09 

-.06 

-.12 

-.16 

i 

o 

-'j 

-.23** 

-.24** 

o 

1 

-.06 

-.09 

.18* 

-.17 

3. 

Perceptual  org. 

— 

1.0 

-.10 

-.19* 

-.21* 

-.15 

-.24** 

-.39** 

-.10 

-.17 

-.26** 

.23** 

-.29 

4. 

Hick  move  1 

-- 

— 

1.0 

.73** 

.69** 

.53** 

.57** 

.49** 

.62** 

.36** 

.53** 

-.10 

.77 

3. 

Hick  move  3 

-- 

-- 

— 

1.0 

.84** 

.50** 

.57** 

.57** 

.63** 

.57** 

.60** 

-.16 

.82 

6. 

Hick  move  5 

— 

-- 

— 

— 

1.0 

.60** 

.67** 

.67** 

.70** 

.63** 

.64** 

-.22* 

.88 

7. 

Hick  no  move  1 

— 

— 

-- 

-- 

— 

1.0 

.73** 

.66** 

.51** 

.47** 

.50** 

-.01 

.75 

8. 

Hick  no  move  3 

-- 

— 

— 

~ 

— 

1.0 

.78** 

.60** 

.56** 

.50** 

-.13 

.84 

9. 

Hick  no  move  5 

— 

-- 

— 

-- 

-- 

-- 

— 

1.0 

.58** 

.57** 

.47** 

-.20* 

IS 

10. 

Arrows  Down 

— 

-- 

— 

— 

-- 

~ 

— 

— 

1.0 

.90** 

.62** 

-.12 

.85 

11. 

Arrows  Side 

-- 

-- 

— 

— 

— 

— 

— 

— 

— 

1.0 

.56** 

-.19* 

.8! 

12. 

Choice  RT 

~ 

— 

— 

— 

— 

— 

— 

— 

— 

— 

1,0 

-.30** 

.77 

13. 

Mental  Counters 
Test 

— 

-- 

— 

-- 

-- 

— 

— 

— 

— 

— 

— 

1.0 

.21 

Mean 

.00 

.33  66 

311 

396 

431 

271 

322 

353 

487 

546 

558 

27 

SD 

.77 

.81  6.0 

64 

80 

92 

60.8 

61.4 

75 

81 

101 

73 

7.0 

aFactor  loadings  for  first  of  unrotated  matrix  for  Principal  Components  factor  analysis  (total  variance  accounted  for  equals  48.3%).  Salient  loadings  are 
underlined. 

*2  <  .05. 

**£<  .01, 


Table  10 


Varimax  Rotated  Factor  Matrix  for  Computerized  Tests 


Factor  1 
Reaction  Time 

Factor  2 

Visual  Processing  Speed 

Factor  3 
Mental  Counters 

IT-Adaptive 

.25 

.53 

-.33 

IT-Nonadaptive 

.04 

-.72 

.06 

PO  Test 

-.01 

-.62 

.09 

Hick  Move  1 

.82 

-.04 

1 

• 

o 

V/1 

Hick  Move  3 

.82 

.09 

-.13 

Hick  Move  5 

.87 

.15 

OO 

o 

• 

1 

Hick  No  Move  1 

.76 

.23 

.32 

Hick  No  Move  3 

.80 

.36 

.22 

Hick  No  Move  5 

.71 

.49 

.21 

Arrows  Down 

.87 

.03 

-.20 

Arrows  Side 

.82 

.04 

-.24 

Choice  RT 

.71 

.16 

O 

CJ- 

• 

1 

Mental  Counters 

1 

* 

o 

-.24 

.75 

Table  11  presents  the  correlations  between  each  of  the  criterion  variables  and  the 
major  indices  for  the  computerized  tests.  Inspection  of  these  correlations  reveals  that 
the  two  best  correlates  of  intelligence  (as  measured  by  the  Raven)  were  the  Arrows  Tests 
(r  =  -.28)  and  the  MCT  (r  =  .43).  To  get  a  better  picture  of  the  relationship  between  the 
microcomputerized  tests  and  criterion  measures,  the  significant  correlations  in  Table  11 
are  corrected  for  attenuation  into  Table  12. 

Finally,  the  finding  of  significant  improvement  on  several  indices  (see  Table  8)  left 
open  the  question  of  whether  improvement  on  the  microcomputerized  tests  might  reflect 
high  ability.  To  get  at  this  question,  a  difference  score  was  computed  for  each  subject  for 
each  of  the  major  indices  and  correlated  with  each  of  the  criterion  variables.  Table  13 
provides  a  summary  of  this  analysis.  As  the  table  shows,  difference  scores  did  not  relate 
well  to  the  criterion  variables. 
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Table  1 1 


Correlations  for  Criterion  Variables  and  Computerized  Tests3 


P3 

SI 

VZ3 

GRH 

GLH 

Raven 

SATV 

SATM 

HSGPA 

FRGPA 

IT-Adaptive 

-.04 

-.24* 

-.18 

.04 

.29** 

-.07 

.00 

-.13 

-.06 

-.15 

IT-Nonadaptive 

-.02 

.11 

.23* 

.25* 

.11 

.02 

.05 

.11 

-.06 

.14 

PO  Test 

.10 

.27* 

.15 

-.01 

-.13 

.20* 

.21* 

-.03 

.27* 

.16 

Hick  Move  1 

.06 

.10 

-.09 

.22* 

.15 

.04 

.00 

-.24* 

-.04 

-.25* 

Hick  Move  3 

-.11 

-.24* 

-.19 

.10 

.10 

-.10 

.11 

-.13 

-.14 

-.24* 

Hick  Move  5 

-.02 

-.26** 

-.22* 

.03 

o 

o 

♦ 

-.15 

.07 

-.12 

-.16 

-.29* 

Hick  No  Move  1 

.08 

-.07 

-.15 

.09 

-.00 

-.00 

-.10 

-.13 

.13 

-.13 

Hick  No  Move  3 

O 

• 

1 

-.17 

-.11 

cr\ 

O 

1 

.08 

-.11 

-.14 

-.19 

O 

• 

1 

-.27* 

Hick  No  Move  5 

-.01 

-.30** 

-.20* 

-.02 

-.11 

-.15 

-.18 

-.31** 

-.06 

-.26** 

Arrows  Down 

.04 

-.29** 

-.15 

.10 

.10 

-.17 

-.10 

-.23* 

VO 

o 

• 

1 

-.17 

Arrows  Side 

.04 

-.37** 

-.25* 

.03 

.10 

-.28** 

-.14 

-.34** 

-.05 

-.08 

Choice  RT 

-.10 

-.06 

-.26* 

.07 

.28** 

-.07 

.09 

.06 

-.24* 

-.24* 

Mental  Counters 

.02 

.20* 

.48** 

.34** 

.06 

.43** 

.21 

.46** 

.40** 

.14 

Not  corrected  for  attenuation. 

*£  <  .05. 

**£<  .01. 


Table  12 


Correlations  for  Criterion  Variables  and  Computerized  Tests 

for  Attenuation3 


P3 

SI 

VZ3 

GRH 

GLH 

Raven 

SATV 

SATM 

HSGPA 

FRGPA 

IT-Adaptive 

— 

-.4  8 

-- 

-- 

.58 

— 

— 

— 

— 

— 

IT-Nonadaptive 

— 

— 

.31 

.34 

— 

-- 

— 

— 

— 

— 

PO  Test 

— 

.31 

— 

— 

— 

.23 

.25 

— 

.32 

— 

Hick  Move  1 

— 

— 

.28 

— 

— 

— 

-.30 

— 

-.32 

Hick  Move  3 

— 

-.28 

— 

— 

— 

— 

— 

— 

— 

-.28 

Hick  Move  5 

— 

o 

1 

-.25 

— 

— 

— 

— 

— 

— 

-.33 

Hick  No  Move  1 

— 

— 

— 

— 

— 

— 

— 

— 

_ 

__ 

Hick  No  Move  3 

— 

— 

— 

— 

— 

— 

— 

— 

— 

-.33 

Hick  No  Move  5 

— 

-.40 

-.26 

— 

— 

— 

— 

-.41 

— 

-.34 

Arrows  Down 

— 

-.34 

— 

-- 

-- 

— 

-.27 

Arrows  Side 

— 

-.41 

-.28 

— 

— 

-.31 

— 

00 

• 

— 

Choice  RT 

— 

— 

-.30 

— 

.33 

— 

— 

— 

-.28 

-.28 

Mental  Counters 

— 

.26 

.62 

.39 

— 

.56 

-- 

.60 

.52 

— 

Only  those  correlations  that  reached  statistical  significance,  uncorrected  for  attenuation,  are  included. 
All  corrections  are  based  on  the  reliability  coefficients  presented  in  Table  6. 


Table  13 


Correlations  of  Difference  Scores  (Improvement) 
and  Criterion  Variables 


P3 

SI 

VZ3 

GRH 

GLH 

Raven 

SATV 

SATM 

HSGPA 

FRGPA 

IT-Adaptive 

.09 

.24* 

.07 

.07 

.45** 

.01 

.12 

.05 

.08 

-.18 

IT-Nonadaptive 

-.18 

-.00 

.11 

.09 

.08 

.01 

-.20 

.00 

-.31** 

.12 

Perceptual  Org. 

-.20 

-.05 

.25* 

.31** 

-.07 

.07 

.07 

-.09 

.08 

-.12 

Hick  Move  1 

.18 

-.03 

.17 

.15 

.17 

-.06 

-.16 

-.20 

.16 

-.08 

Hick  Move  3 

-.30** 

-.27** 

-.20* 

-.03 

.06 

-.01 

.22 

-.01 

.10 

.13 

Hick  Move  5 

-.04 

-.00 

.04 

.21 

-.10 

-.14 

.04 

.17 

-.10 

.10 

Hick  No  Move  1 

.05 

.04 

.01 

.21 

-.06 

.16 

-.10 

-.00 

.27* 

ir\ 

O 

• 

1 

Hick  No  Move  3 

-.13 

-.06 

.25* 

.03 

.06 

.02 

.14 

.13 

.12 

.18 

Hick  No  Move  5 

-.10 

-.15 

.08 

.30** 

-.06 

-.01 

-.03 

-.09 

.02 

.16 

Arrows  Down 

-.20 

-.28* 

.00 

.09 

.04 

.02 

.08 

O 

o 

• 

.04 

.16 

Arrows  Side 

-.18 

-.25* 

-.07 

-.07 

.11 

-.07 

.02 

-.03 

.03 

-.01 

Choice  RT 

-.10 

.27* 

-.06 

-.25* 

.12 

00 

o 

• 

.18 

1 

• 

o 

ON 

.29* 

.10 

MCT 

.17 

.03 

.20 

.25* 

-.06 

.22* 

.12 

.15 

.03 

-0.08 

*£  <  .05. 

**£<  .01. 


DISCUSSION 


Overall,  the  results  support  the  continued  development  of  cognitive  speed  tests  by 
demonstrating  that  reliable  individual  differences  exist  on  certain  versions  of  such  tests. 
The  results  also  verified  the  previously  reported  distinction  (e.g.,  Saccuzzo,  Larson,  & 
Rimland,  1986;  Vernon,  1983)  between  measures  of  RT  and  IT.  RT  and  IT  tasks  loaded  on 
separate  factors  and  had  a  different  pattern  of  correlation  with  the  various  criterion 
variables. 

Based  on  our  results  the  following  tests  deserve  further  development: 

Inspection  Time--The  IT-nonadaptive  test  with  a  horizontal  line  orientation  was 
clearly  the  most  reliable  "line  discrimination"  task.  The  task  could  be  further  improved 
by  boosting  the  number  of  trials  from  50  to  75.  The  estimated  test  time  would  be 
approximately  15  minutes,  which  should  be  operationally  feasible.  Further  work  on  an  IT- 
adaptive  test  could  follow  calibration  of  item  difficulties  on  a  large,  random  sample  of 
subjects. 

Considering  the  more  established  IT  tasks  and  the  new,  experimental  Perceptual 
Organization  IT  task,  the  latter  proved  to  be  the  most  stable.  It  yielded  test-retest 
coefficients  of  .73  (see  Table  6)  and  split-half  coefficients  of  .67  to  .79  (see  Table  7). 
Additional  work  with  this  test  is  indicated. 

In  conclusion,  further  research  on  IT  should  include  a  75-item  nonadaptive  test  with 
horizontal  test  lines  and  the  Perceptual  Organization  IT  test. 

Reaction  Time— In  considering  the  RT  tests  as  part  of  a  computerized  battery,  it  is 
noteworthy  that  they  are  highly  intercorrelated,  load  on  the  same  factor,  and  are  related 
to  the  same  criterion  variables.  Thus,  a  wide  battery  of  such  tasks  is  not  needed.  Rather, 
an  indicated  strategy  would  be  to  select  one  or  two  of  the  best  of  these  paradigms.  Data 
presented  herein  indicate  that  the  two  most  stable  of  these  are  the  Hick  paradigm  with 
movement  and  the  Arrows  test.  Since  the  Hick  paradigm  requires  a  particular  keyboard 
configuration  and  the  Arrows  test  does  not,  the  latter  is  better  suited  for  administration 
on  the  military's  proposed  first  generation  computerized  testing  hardware  (Hewlett 
Packard  Integral  computers),  which  has  a  customized  keyboard  with  relatively  few 
response  keys.  Continued  research  on  the  Arrows  test  is  recommended. 

Mental  Counters— The  results  with  the  MCT  are  also  noteworthy.  The  MCT  had  a 
test-retest  reliability  of  .59,  with  split-half  coefficients  of  .68  and  .79.  In  addition  to 
being  somewhat  stable,  this  new,  experimental  test  loaded  on  its  own  factor,  indicating 
the  measurement  of  a  source  of  variability  not  being  tapped  by  either  RT  or  IT  tasks.  The 
MCT  also  produced  some  of  the  strongest  relationships  with  criterion  variables, 
correlating  (corrected  for  attenuation)  .62,  .56,  .60,  and  .52  with  the  VZ3  test,  the  Raven, 
SATM,  and  HSGPA,  respectively.  If  present  results  can  be  confirmed,  the  MCT  may  prove 
to  be  an  excellent  choice  for  inclusion  in  a  well-rounded  battery  of  computerized 
cognitive  tasks. 

,  1*  is  important  to  emphasize  the  limitations  of  the  present  investigation.  First,  all  of 

the  tasks  were,  by  design,  nonverbal.  The  reliabilities  for  verbal  computerized  tasks  were 
not  evaluated  and  may  be  different  from  those  obtained  herein.  Second,  the  data  were 
collected  on  a  college  sample  of  104  subjects  (test-retest  coefficients  are  based  on  an  N 
of  74).  Results  may  not  be  fully  generalizable  to  other  settings  where  tests  are  widely 
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administered,  such  as  the  Armed  Forces.  Replication  of  the  present  results  on  varied 
populations  is  therefore  advisable. 


CONCLUSIONS 

The  following  conclusions  appear  to  be  warranted  from  the  data. 

1.  RT  and  IT  tests  load  on  separate  factors,  and  may  have  different  patterns  of 
validity.  Certain  tests  of  each  construct  have  sufficient  reliability  to  justify  further 
development. 

2.  RT  tasks  of  all  types  are  highly  intercorrelated  and  related  similarly  to  a  variety 
of  criterion  variables.  Consequently,  it  may  not  be  necessary  to  have  a  wide  battery  of 
such  tasks--one  or  two  well-selected  tasks  are  all  that  is  needed.  Of  the  RT  tests 
administered  in  the  present  study,  the  Arrows  test  is  the  best  prospect  for  further 
research. 


3.  Of  the  IT  tasks,  those  involving  horizontal  lines  in  a  nonadaptive  paradigm  are  by 

far  the  most  reliable.  Increasing  the  number  of  trials  from  50  to  75  would  result  in  a 

better  test.  A  nonadaptive,  75  item  horizontal  line  test  and  the  newer  Perceptual 

Organization  IT  test  both  deserve  further  evaluation. 

4.  The  MCT  is  particularly  promising  in  that  it  loads  on  its  own  factor  and  thus 

appears  to  be  tapping  into  a  source  of  variance  different  from  RT  and  IT  tasks. 

5.  The  continued  development  and  validation  of  measures  of  RT,  IT,  and  other 
variables  appears  to  hold  considerable  promise  for  measuring  aspects  of  ability  not  tapped 
by  conventional  psychometric  tests. 
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